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ABSTRACT: In this paper, we are establishing many interesting and important relations among several 
divergence measures by using known inequalities. Actually this work is application of well known 
inequalities in information theory. Except various relations, we tried to get bounds of 

Nl(P,Q),r k (P,Q),A k (P,Q),El(P,Q),S*(P,Q),L(P,Q), ¥ M(P,Q),R 2 (P,Q) in 

terms of standard divergence measures. Some relations in terms of Arithmetic Mean A^P, <2), Geometric 

MeanG (i 3 , <2)> Harmonic Mean H (i 3 , <2) , Heronian MeanN {^P,Q^, Contra Harmonic Mean 

C^P, <2) , Root Mean Square S (P,Q^ and Centroidal Mean R^P, <2), are also obtained. 
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I. Introduction 

Let T n = j,P = (p l ,p 2 ,p 3 , p n ) : p j > 0, ^ Pj = 1 j , n > 2 be the set of all complete finite discrete 

probability distributions. If we take /? ; >0 for some i = 1, 2, 3, n , then we have to suppose that 

0/(0) = 0/ - = 0 . Csiszar's f- divergence [1] is a generalized information divergence measure, which is 
given by: 



c /(^e)=2>,/ 



(1) 



Where f: (0,oo) — > R (set of real no.) is a convex function and P, Q e T n . Many known divergences can 
be obtained from these generalized measures by suitably defining the convex function f. 
By (1), we obtain the following divergence measures: 
Following measures are due to (Jain and Srivastava [7]). 

&(Pi-4i) 



■ , k — 1, 3, 5, 7, 



i=1 (Pi*)* 



Jl{P,Q) = t iE ^^ {Pt - qir 



m ( A ^.) 2 [ P t q t 
Following measures are due to Kumar P. and others. 

i-i P,1, {2sIp,% 



,k — 1, 3, 5, 7, ... 



(Kumar P. and Chhina [9]) 



(2) 



(3) 



(4) 
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I 2 2\ 2 
- \ P — Q I 

y/M (P, Q) = X '-j- (Kumar P. and Johnson [11]) 

L(P,G) = X^^lo, 
f=f p, + q, 



f \ 
Pt + % 



1, 



(Kumar P. and Hunter [10]) 



2^ 

Renyi's second order entropy (Renyi A [12]). 

i=l % 

Puri and Vineze Divergence Measures (Kafka,Osterreicher and Vincze [8]). 



b\Pi-Qi\ 



\k+l 



-,fce(0,oo) 



'=i [Pi + li) 

Relative Jensen- Shannon divergence (Sibson [13]). 

F(P,Q) = Y J P,log{^- 
Relative Arithmetic- Geometric Divergence (Taneja [14]). 



log 



2p 



i J 



Arithmetic- Geometric Mean divergence Measure (Taneja [14]). 



1 



r(p, G )^[G(p,G) + G(G,p)] = X^og 



Pi+<ii 



(=i 



2 i 



Pi q 



i ~Li J 



Where G(P,Q)is given by (10). 

Symmetric Chi- square Divergence (Dragomir, Sunde and Buse [4]). 

V(P,Qh± {Pi ~ qi)2{Pi + qi) 

«=i Pi q 

Relative J- Divergence (Dragomir, Gluscevic and Pearce [3]). 

J R (P,Q) = 2[F(Q,P) + G(Q,P)] = f j (p,-q i )log 

;=i 

Where F (P, Q) and G(P, Q) are given by (9) and (10) respectively. 
Hellinger Discrimination (Hellinger [5]). 

Triangular Discrimination (Dacunha- Castelle [2]). 



Pi+q 



2q, 



A(P,Q) = ii^ 
tf Pi + q 



Except above, we obtain the following divergence measures (Due to Jain and Saraswat [6]). 

^(P,e) = S^Cexp|(i^4u = l,2,3,... 



,=i (p,+q,) 



(Pi+qd 



(5) 



(6) 



(7) 



(8) 



(9) 



(10) 



(11) 



(12) 



(13) 



(14) 



(15) 



(16) 
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II. Well Known Inequalities 

The following inequalities are famous in literature of pure and applied mathematics, which are important tools 
to prove many interesting and important results in information theory. 

l + t <e < l + te', t>0 (17) 



— <log(l + t)<t, t>0 
l + t 



(18) 



III. Relations Among Various Divergence Measures 

Now, we shall obtain bounds of some measures in terms of other divergence measures and many important and 
interesting relations among several divergence measures by using inequalities (17) and (18) respectively. 
Proposition 1: Let (P,Q)er n xr n , then we have the inequalities: 

N;(P,Q)-Nl +l (P,Q)<A 2kl (P,Q) 



And A 2k+l (P,Q)<Nl +l (P,Q) 

Where k =1, 2, 3, and TV* (P, Q) , A k (P,Q) are given by (16) and (8) respectively. 



(19) 
(20) 



Proof: Put t 



in inequalities (17), we get 



: exp 

(Pi + 9 t ) 
now multiply the above expression by 



(Pi + Qi) (Pi + Qi) (Pi+li) 



{Pi + Vi) 



k — 1, 2, 3, ... and sum over all i=l, 2, 3...n, we get 



Lr, + — ex P 



1=1 



(Pi ~ «=i (Pi +q t ) 



^(Pi+ii) 
(Pi-q t ) 



(Pi+9 t )\ 



exp 



i=1 [Pi +<?;) ^{Pi+%) [{Pi+%) 

i. e. A 2k , (P, Q) + A 2M (P,Q)< Nl {P,Q)< A 2 ^ (P, Q) + N* k+1 (P, Q) 

From second and third part of (21), we get inequality (19) and from first and third part, we get (20). 
Now at k=l, 2, 3 ... we get the followings [from inequalities (19) and (20)]: 

Atk=l => {P,Q)-N* 2 {P,Q)<A (P, Q) and {v A x (P,Q) = A(P,Q)} 

a 3 (p,q)<n;(p,q) 

Atk=2_^ A^ (P, Q) - Nl (P, Q) < A 3 (P, Q) and 

a 5 (p,q)<n;(p,q) 

Atk=3 ^ N* 3 (P, Q) - N* 4 (P, Q) < A 5 (P, Q) and 

A 7 (P,£)<A;(P,£) and so on... 
Proposition 2: Let (P,Q)er n xr n , then we have the inequalities: 

J* k (P,Q)-Jl 2 (P,Q)<E\{P,Q) 
And E\ +2 (P,Q)<Jl 2 (P,Q) 

Where k = 1, 3, 5, <md E* k (P,Q), J* k (P,Q) are given by (2) and (3) respectively. 



(21) 



(22) 



(23) 
(24) 
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(p ~Q ) 

Proof: Put t = in inequalities (17), we get 

Pi ft 

. ^ {Pi - <ii y . {Pi - % y , ! ^ (Pt - ii y (Pi - <?, y 

(Aft) ' ' 

Now multiply the above expression by 



(p, ft) (p, ft) (P,ft) 

(Pi-Qt) 



(Pi ft) 



k/2 



, k = 1, 3, 5, ... and sum over all i=l, 2, 3...n, we get 



y lA-ft) ,y A~ft ^ y (A ~ft) 



(Aft) « (ft ^2 



£+1 



^y A~ft L y (A-gj 

^2.^ ^ + 2. r- ex P 

M (Aft) w ( p . q.)2 +l 



(Pi ft) 

(a -ft) 2 
(a ft) 



(a - ft) 2 
(a ft) 



(25) 



i. e. E* k (P,Q) + E* k+2 (P,Q)< f k (P,Q)< E[ (P,Q) + f k+2 (P, Q) 

From second and third part of (25), we get inequality (23) and from first and third part, we get (24). 
Now at k=l, 3, 5 ... we get the fallowings [from inequalities (23) and (24)]: 

Atk=l_=> J* (P, Q) - ^ (P, Q) < El (P, Q) and 

E\(P,Q)<Jl(P,Q) 

Atk=3_^ Jl (P,Q)- J* 5 (P, Q) <E\{P,Q) and 

E\(P,Q)<Jl(P,Q) and so on... 
Except these, from first and second part of the inequalities (25), we can easily see that 

El(P,Q)<Jl(P,Q) 

Proposition 3: Let (P,Q)er n xr n , then we have the inequalities: 
¥ (P,Q)-2El(P,Q)<S*(P,Q) 

And S*(P,Q) + y/(P,Q)<y/M(P,Q) 

Where X//(P,Q), El(P,Q), S* (P,Q) and y/M (P, Q) are given by (12), (2), (4) and (5) respectively. 
Proof: Put t = . in inequalities (18), we get 



(26) 



(27) 
(28) 



2 ^/Aft 

(4p-^) 
2 4m 

(Jpl-yftk) 



<log 



1 + 



1 + 



(4p~4^) 

2 ^/Aft~ 



(4p-^) 2 
2 4pi^i 



Pi +ft " 2 >/Aft" 



A +ft 



<log 



Pi +ft 



A +ft ~ 2 4pM 



2 4p7h) 2 4p& 
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Now multiply the above expression by ^— ^' - and sum over all i=l, 2, 3 . . .n, we get 



Pft 



£ {Pi + <ii){Pi-<ii) p, + c ii- 2 4m (A+jO(A -ft) kr . 



2 A A 

Pi + ft 



i=] 



Pft 



Pift 



2 >/p¥ 



< 



J (Pi + * :)(Pi " ft ) 2 Pi + ft " 2 >/pft~ 



Pift 



2 4m 



i.e. 



^ (p ■ ~ ft ) 2 (Pi + ft ) ^ (Pi ~ ft ) 2 ^ 5 * ^ ^ ^ ^ (P.' ~ ft' ) 2 ^ (Pi ~ ft ) 2 (Pi + ft ) 

M Pi ft M > /Kft = ' i=l 2 (p. $)1 i=1 Pi(li 

i. e. ^ (P, Q) - 2E\ (P,Q)< S* (P,Q)< ¥ M(P,Q)- ¥ (P, Q) 

From first and second part of (29), we get inequality (27) and from second and third part, we get (28). 
Except these, if we add (27) and (28), we get the following 

2 y, (P, Q) < ¥ M (P, Q) + 2 E* x (P, Q) 
From second and third part of the inequalities (29), we can easily see that 

S*(P,Q)<ysM(P,Q) 
By taking both (27) and (31), we can write 

y/{P,Q)-2El (P,Q) < S"" (P,Q) < y/M(P,Q) 

Proposition 4: Let (P,Q)er n xr n , then we have the inequalities: 

l(p,q) + a(p,q)<±-e;(p,q) 



And A(P,Q)<L(P,Q) + 2±^( Pi ^ 



(Pi+ft) 

Where L(P,Q), E* (P,Q), A(P,g)are given by (6), (2) and (15) respectively. 
Proof: Put t = „ in inequalities (18), we get 



(29) 

(30) 
(31) 
(32) 

(33) 
(34) 



2 ^/pft = 

Pi + ft " 2 >/pft~ 



Pi + ft 



<log 



Pi + ft 



Pi + ft - 2 4pm> 



2 ^/p^J 2 4m 



( Pi — ft ) 

Now multiply the above expression by — - - — and sum over all i=l, 2, 3...n, we get 



Pi + ft 



£ (p, -ft) 2 P - +ft ~ 2 ^/pft~ < y (Pj-ft) k) T 



if \ 
Pi +Qi 



p+ft 



Pi+Qi 



M Pi + ft 

*2 



2 ^ 



Pift 



< 



J (P, "ft) P, +ft - 2 ^/pft" 



P+ft 



2 4m 



f (p. -ft) 2 2 f JEKihZ^ <u PjQ \<Lf ^ihZ^l y (p~^) 2 

m Pi+ft ;=i (Pi+ft)' ' 2 '=! ^/pi^ *=i P+ft 



1 „* 



i. e. A(P,Q)-2X ^ (A f'> < L(P, G ) < ±l£ (P,G)- A(P, G ) 
M (A+ft) 2 



(35) 
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From second and third part of (35), we get inequality (33) and from first and second part, we get (34). 
From inequality (33), we can easily see that 

a(p,q)<\e;(p,q) 



(36) 



Proposition 5: Let (P,Q)eF n xF n and \\ p i = = 1 , then we have the inequalities: 

A(P,Q)<h(P,Q)<T(P,Q) 

A(P,Q) + h(P,Q)<\t { -^£- 

4 M ylPiQi 



(37) 



And 



A(P,Q) + T(P,Q)<\±iPp^ 

4 M yjp i q i 



(38) 



(39) 



« n + CI 

Where T(P,Q), /j(P,g)are given by (11) and (14) respectively and A(P,Q) = ^ — — — = 1 is well 



known Arithmetic Mean Divergence. 



Proof: Put t 



(4p-4^) 

Pi +q t ~ 2 4m 



in inequalities (18), we get 

Pi ~ 2 4pm 



Pi + q t 



<log 



f \ 

Pi +q t 



2 4p& J 2 ^/m 



Pi + <li 

Now multiply the above expression by and sum over all i=l, 2, 3 . . .n, we get 



h (gi + <h ) Pi + <?, ~ 2 4m < y (gilji) 



Z 



-log 



2^ 



< 



J [Pi+qj) Pi+qj- 2 4p^i 



i. e. 



»{sfp~-4q~) . . "(p.+q.) 2 

MP,G)<r(p,G)<X^i-i 



(40) 



From first and third part of (40), we get inequality (38) and from second and third part, we get (39). 
Except these, from (38) and (40), we can easily see the followings 

4 M Jpfli 
4 M VA^i 

And h(P,Q)<T(P,Q) 

Now do (41)-(42), we get 



(42) 
(43) 
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A(P,Q)-h(P,Q)<O^A(P,Q)<h(P,Q) 

By taking both (43) and (44), we get the inequalities (37). 

n n 

Proposition 6: Let (P,Q)er„xr n and £_j Pi = ^<7, = 1 > men we nave me inequalities: 

i=l i=l 

G(Q,P)>^-\og2 
And log 2 + G (Q, P) < l - [R 2 (P, Q) + 1] 

Where R 2 (P, Q) , G(Q, P) are given by (7) and (10) respectively. 
Pi 

Proof: Put t = — in inequalities (18), we get 

It 



(44) 



(45) 
(46) 



<log 



< 



P 



V Hi J Hi 



Pi + <?, 

Now multiply the above expression by and sum over all i=l, 2, 3 . . .n, we get 



Z 



P,+1, P, ^yMiipg 



2 p.+q. 



m 2 p.+# ; tf 2 ^\2 ^ 



< 



!=1 



z- 



log 



2$ 



Pi 

2 



<Z— +Z- 

tf2^ tT2 



(47) 



=1 i=l 



i. e. I < log 2 + G (Q, P) < 1 [tf 2 (P, g) + 1] 

From first and second part of (47), we get inequality (45) and from second and third part, we get (46). 

n n 

Proposition 7: Let (P,Q)er„xr n and ^ p. = / j q l = 1 , then we have the inequalities: 

log2- F(P,Q)<A(P,Q) 
And ±H(P,Q) + F(P,Q)<log2 

Where F(P,Q) is given by (9), A(P,Q) = ^^^ = 1 and H (P,Q) = ^^^- are Arithmetic 

i=l 2 M /7 ; + g ( . 

Mean and Harmonic Mean Divergences respectively. 
Proof: Put t = — in inequalities (18), we get 

Qi 



(48) 



(49) 



Pi+Qi 



<log 



\ 



Pi + 9i 



< 



Pi 



Now multiply the above expression by 2q t and sum over all i=l, 2, 3 . . .n, we get 



5>,-^<£2tf,log 



2 p.+g. 



2 g 



Le. H(P,e)<21og2X9,-2X9ilog 



^Z^ 



2q t 



Pi+q, 



< 



2^Pi 



i J 
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i.e. H(P,Q)<2\og2-2F(Q,P)<2 

After interchanging P and Q, we get the following 

H(P,Q)<2\og2-2F(P,Q)<2 (50) 

from second and third part of (50), we get inequality (48) and from first and second part, we get (49). 
Some Relations: 

vH(P,Q)<G(P,Q)<N(P,Q)<A(P,Q)<R(P,Q)<S(P,Q)<C(P,Q)(T a nw 

The above inequalities (51) is a famous relation among seven means, where 

H(P,Q),G*(P,Q),N(P,Q),A(P,Q),R(P,Q),S(P,Q),C (P, Q) are mentioned in abstract. 

Now we can get some other important relations among various divergences with the help of above inequalities, 
these are as follows. 

❖ from (37) and (5 1 ), we get 

H (P, Q) < G* (P, Q) < N(P, Q) < A(P, Q) <h(P,Q)<T (P, Q) (52) 

❖ from (48) and (5 1 ), we get 

log 2- F(P,Q)<A(P,Q)<R(P,Q)<S(P,Q)<C(P,Q) (53) 

❖ from (37) and (48), we get 

log 2 - F (P, Q) < A (P, Q) < h (P, Q) < T (P, Q) (54) 

❖ do (46) - (48), we get 

G(Q,P) + F(Q,P)<^[R 2 (P,Q) + l]-A(P,Q) 
i. e. 2A(P, Q) + 2[G(Q, P) + F (Q, P)] < R 2 (P, Q) + 1 

i.e. 2A(P,Q) + J R (P,Q)<R 2 (P,Q) + l (55) 

❖ from (22), (26) and (36), we get 

N';(P,Q)-N;(P,Q)<A(P,Q)<^E;(P,Q)<^J;(P,Q) (56) 
♦:♦ from (22) and (23), we get 

N* 1 (P,Q)-N* 2 (P,Q)<A(P,Q)<^El(P,Q)-L(P,Q) (57) 
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